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classified as moderate, and the Arts strand was classified as high in 
implementation. Using the list of goals the steering committee originally set 
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(Arts, Music, and Foreign Language) would be considered high, implementation 
in Family Involvement, Health/Emotional Health, and Thinking Skills would be 
considered moderate, and implementation in Reading and Language Development 
would be considered low. The degree to which these results agree with the 
estimation of implementation by the district superintendent is discussed 
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Abstract 

This report is about the third year implementation of Graduation 2010, a district wide 
public school initiative that had its beginnings in brain based research and involvement from 
both school and community. The original plan for the program had eight strands: Arts, Music, 
Foreign Language, Reading/Language Development, Thinking Skills, Health/Emotional Health, 
Family Involvement, and Community Involvement. Surveys, similar to those used in the 
previous year, were distributed to principals in the 12 elementary schools and to the 
superintendent. These administrators were asked to rate the implementation of the original goals 
of the project from 'O' (no implementation) to '5' (full implementation). Based on survey results, 
implementation was classified as low, moderate, or high for each strand. Based on the complete 
list of survey items, two strands, Health/Emotional Health and Thinking Skills, were classified as 
low in implementation. Music, Reading/Language Development, Foreign Language, and Family 
Involvement were classified as moderate, and the Arts strand was classified as high in 
implementation. Using the list of goals the steering committee originally set for the 1997-98 
school year as the standard, implementation in three strands (Arts, Music, and Foreign 
Language) would be considered high, implementation in Family Involvement, Health/Emotional 
Health, and Thinking Skills would be considered moderate, and implementation in Reading and 
Language Development would be considered low. The degree to which these results agree with 
the estimation of implementation by the district superintendent is discussed along with 
precautions and problems of interpreting these results in general. 
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Graduation 2010: Third Year Implementation 
Graduation 2010 is a district wide public school initiative that resulted from the 
cooperative planning efforts of concerned educators and citizens of the Daviess County 
community in Daviess County, Kentucky. The group researched relevant information on brain 
development and formed committees to study ways to improve public school education for the 
students in the county. The primary goal of the project was to increase the intellectual capacity 
of their students. The result was the formation of eight subcommittees and a set of 
recommendations with implementation guidelines from each. The eight committees were Music, 
the .Arts, Foreign Language, Thinking Skills, Family Involvement, Health/Emotional Health, 
Reading/Language Development, and Community Involvement. A review of the empirical 
evidence used by each subcommittee and the recommendations of the subcommittees for each 
strand have been presented in previous reports on the project (see Norman, O'Phelan, & Ecton, 
1998; O’Phelan, Norman, & Ecton, 1999b). 

Briefly, recommendations of subcommittees included establishing keyboard labs for 
Music, coordinating arts experiences for the Arts, implementing a Spanish curriculum in Foreign 
Language, implementing Talents Unlimited (Chissom & McLean, 1993) and chess for Thinking 
Skills, providing student orientations and establishing homework committees for Family 
Involvement, providing nursing services and fitness programs for Health/Emotional Health, and 
establishing a program to help primary children who are reading below grade level for Reading 
and Language Development. The goal of the Community Involvement committee was to secure 
a corporate sponsor from the community for each new kindergarten class. 

During the first year of Graduation 2010, the research team met several times with 
administrators and steering committee members to plan an evaluation strategy that would be 
responsive to the needs of the program. Several research questions resulted (see O'Phelan, 
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Norman, & Ecton, 1999a for a discussion of these). The first question addressed the degree of 
implementation of the program by the schools. To answer this question, we compared the 
original goals and steps for implementation of the project with practice in the schools. Reports 
on the implementation during the first two years of Graduation 2010 are available in Norman et 
al., 1998 and O’Phelan et al., 1999b. This report is about the implementation during the third 
year of the project. 



Method 

Each subcommittee (except Community Involvement) submitted a list of long term goals 
and short term goals to be accomplished during the first year of the program (the 1997-98 school 
year). A list of original goals is presented in the appendix of O’Phelan et al., 1999b. During 
these three years of implementation, we have gathered information with surveys based on the 
implementation steps provided by the original subcommittees. 

Survey Instrument 

The original list of implementation steps varied in number from a minimum of three in 
the Arts strand to a maximum of 30 in the Family Involvement strand (see column one, Table 1 
below), and in some cases implementation steps were written such that they included various 
parts. (The Community Involvement strand is not included in this discussion since the 
subcommittee had one recommendation, and no implementation steps.) The list of 
implementation steps, or goals was used to construct survey items for the evaluation of 
implementation at the end of the first year (see Norman et al., 1998). Thus, in this report, the 
term 'steps' refers to original subgoals of committees or subcommittees, and the term 'items' 
refers to actual questions appearing on the surveys we developed. To accommodate the dynamic 
aspect of the project, a modified principal survey was used at the end of the second year. The 
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modifications are described in detail in the second year implementation report (O’ Phelan et al., 
1999b). The survey used for the third year also underwent some modifications. Survey items 
were submitted to district personnel and to the coordinator of Graduation 2010 for revision 
suggestions. Some survey items were eliminated because it was decided not to implement them, 
and the wording was changed slightly in others to reflect the current intent of the program after 
three years in operation (e.g., an item which originally read “Spanish curriculum presented to PI 
in second year,” was modified to read “Spanish curriculum extended to K-5” since children in 
kindergarten received the curriculum the first year, PI the second year, etc.). There are some 
substitutions in the survey items because elements in the programs offered have changed (e.g., 
other reading programs have replaced the original Kentucky Reads program, so the original steps 
of the reading goals having to do with implementing Kentucky Reads no longer apply). Finally, a 
few items were added to the existing list of survey items. 

Table 1 presents an overview of the number of items on the principal survey by year. 

The number of implementation steps in the original plan for the project appears in column one. 
Column two shows the number of steps in the 1997-98 list of goals, and column three and four 
give the number of steps included in the 1999 and 2000 versions, respectively. Similarity of 
numbers across columns does not necessarily mean that the items on the surveys from one year 
to another were identical. The items on the 2000 survey were similar in number and content to 
those on the 1999 survey. A list of the items as they appeared on the survey is included in the 
appendix. 
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Table 1 

Number of implementation steps by strand according to the original plan of Graduation 2010, in 
the 1997-98 goals list, and represented on the 1998, 1999, and 2000 surveys of principals 





No. of 
original 
steps 


No. of 
steps 
1997-98 
goals 


No. of 
items 
1998 
survey 


No. of 
items 
1999 
survey 


No. of 
items 
2000 
survey 


Strand 












Arts 


3 


3 


3 


6 


6 


Family Involvement 


30 


7 


8 


18 


21 


Foreign Language 


8 


2 


8 


3 


3 


Health/Emot. Health 


25 


2 


8 


5 


6 


Music 


24 


4 


19 


21 


21 


Reading/Lang. Dev’mt 


13 


1 


4 


2 


2 


Thinking Skills 


9 


2 


4 


7 


7 



Implementation Survey Procedures 

At the end of the third year, during the summer, the 2000 surveys were numbered and 
distributed to all 12 elementary school principals. Principals were not required to provide 
identification of their schools on the surveys. Participants were asked to report the degree of 
implementation in their schools on a scale from 0 (no implementation) to 5 (full implementation) 
for each item listed on the survey. Surveys were collected by district personnel and transferred 
to the evaluation team. 
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Results 

Implementation Survey - End of year three; all goals: 

All 12 elementary principals reported at the end of the third year. In Tables 2 through 8, 
the end of the year survey results are presented by survey items for each strand of Graduation 
2010. The reported implementation results for the Arts strand are shown in Table 2. 
Implementation in the Arts strand was high with 12 principals saying they had fully implemented 
3 of the 5 items on the 2000 survey. All schools had an arts facilitator, were providing nine 
experiences in the arts (see Table 2, item 2), and Riverpark experiences for kindergarten through 
P3 classes (item 3). Ten schools were providing at least 20 hours of instruction in grades 4 and 5 
(item 5). Seven schools reported that they had had professional development in the area of the 
arts. When asked what other arts activities the school had had, 1 1 of the 12 principals responded 
with a variety of things (see Table 2: comments, item 7). The comments would seem to validate 
the high rating for implementation in the Arts. The superintendent's rating of implementation in 
the Arts was a 4 out of a possible 5. 

Table 3 contains the reported implementation for the Family Involvement strand. Over 
all items, the number of schools reporting full implementation ranged from one to 12, with at 
least some implementation on every step, and more progress on items 1 (improve first contact 
with the school), 3 (make kindergarten registration special), 19 (provide more opportunities for 
families to visit school), and 22 (include student planners). At least half the principals reported 
full implementation on 10 additional items (see Table 3). Comments made by principals on 
these items are included in Table 3. 
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Table 2 

Number of schools (n = 12, unless otherwise noted) at the end of year three reporting at each 
level of implementation for survey items about the Arts where 0 was no implementation, 1 was 
very little implementation, and 5 was fully implemented and functioning. Principals’ comments 
on Arts items are included. 



Level of implementation 

Strand Survey items 0 1 2 3 4 5 

Arts: 



1 . Identify arts facilitator 


0 


0 


0 


0 


0 


12 


2. Provide 9 arts experiences 


0 


0 


0 


0 


0 


12 


3. Provide Riverpark experience for K 


0 


0 


0 


0 


0 


12 


5. 20 hours instruction, grades 4 & 5 n=l 1 


0 


0 


0 


1 


0 


10 


6. Professional development 


0 


0 


3 


1 


1 


7 



Comments Item : 

3. -all classes attend at least one performance 

5. -did not provide instruction this year, we have in the past 
-instructor taught art to 4 th and 5 th 40 minutes per week per class 

6. -curriculum alignment-purchased visual arts textbooks for 5 th grade 

- trip to fine art museum for faculty meeting 

7. What additional activities have you had for the Arts? (new item) 

- Owensboro Symphony, guest artist 

- Creative connections grant for graphic artist K-5. S. E. from Fine Arts 
Museum 

- Renaissance Faire: all grade levels 

- S. has helped with our cultural awareness celebrations 

- Wolfe Brothers concert on May 17 th , artist made kites with students in grades 
P2-5 ,h on May 12 th , students used art works from Fine Art Museum 

- peer docents program, musical plays, parent teaching art 

- speech contest, weaving musical, pottery 

- Flarvest Day, Discovery Day, Discovery Team, Art Sense all provide 
additional art activities 

- Many —worked very closely with S.E. from Fine Arts Museum on school wide 
project 

- we did far more than 9 experiences at every grade level. Included several 
museum visits-OMFA and Shield, Symphony, RiverPark 

- assemblies, drama, music, RiverPark mini grant for artist to work with 4-5 
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Table 3 

Number of schools (n = 12, unless otherwise noted) at the end of year three reporting at each 
level of implementation for survey items about Family Involvement where 0 was no 
implementation, 1 was very little implementation, and 5 was fully implemented and functioning. 
Principals’ comments on Family Involvement items are included. 



Level of implementation 



Strand 


Survey items 


0 


1 


2 


3 


4 


5 


Family 

Involvement 


1 . Improve first contact, picnic 


0 


0 


0 


0 


1 


11 




2. Distribute packets 


1 


1 


2 


1 


0 


7 




3. Special K. registration 


0 


0 


0 


0 


0 


12 




4. Personal notes to students 


0 


1 


3 


3 


1 


4 


* 


7. Homework committee/philosophy 


1 


0 


0 


1 


3 


7 




8. Specific instructions re homework 


0 


0 


0 


1 


2 


9 




8a. Provide syllabus if needed n=4 


0 


0 


0 


2 


1 


1 




9. Reevaluate amount of homework 


0 


0 


1 


1 


4 


6 




12. Coordinator to PTO board 


2 


0 


0 


0 


2 


8 




13. Teacher involvement in PTO n=l 1 


0 


0 


0 


0 


4 


7 




19. More opportunities for family visits 


0 


0 


0 


1 


1 


10 




22. Include student planners 


0 


0 


0 


0 


1 


11 




23. Improve written communication n=l 1 


0 


1 


1 


1 


2 


6 




23a. Interim progress reports 


0 


0 


0 


2 


2 


8 




23b. Personal notes 


0 


0 


0 


4 


4 


4 




23c. Teacher articles in paper n=l 1 


2 


0 


2 


3 


1 


3 




26. Schedule meetings: parents & teachers 


0 


0 


0 


1 


3 


8 




27. Public relations person 


0 


1 


0 


3 


1 


7 




29. Welcoming Schools criteria n=l 1 


0 


1 


1 


2 


4 


3 




30. Mentoring for parent involvement 


3 


5 


0 


1 


1 


2 




31. Professional development 


2 


7 


2 


0 


0 


1 



Comments Item: 

7. -site based council has worked on homework issues extensively this year 



12. -we have had over 3000 hours of volunteer work annually and parents are 

quite involved. A volunteer coordinator works with staff and PTO 
-we have a volunteer coordinator in this position 

13. -teacher now serves on PTO board 

-teachers work in booths at two family fun nights 

23. -team newsletters 

26. -we have tried every type of scheduling we can think of and have even sent 
special invitations to parents to attend--NO INCREASE 
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When asked to mention any new opportunities for families to visit school (item 19), 
principals mentioned between 3 and 7 activities each (see Table 4). Activities mentioned, in 
descending order of frequency, included curriculum related or special events (n = 34), events to 
recognize individuals (n = 20), and having volunteers in school (n = 5). The overall rating for 
implementation of Family Involvement could be called moderate. The superintendent's rating of 
implementation in Family Involvement was 4 out of a possible 5. 



Table 4 

Family participation activities, arranged by category and by frequency mentioned by principals 



Category 
of Activity 


Examples: 


Times 
activity was 
mentioned 


Curriculum 
related & 
Special 
events 


Family reading night (n=9), class performance/play (n=5), talent 
show n=4), Manners Day (n=3), Field Day (n=2), Harvest Day 
(n=2), Book Fair, Mexican Festival, Cultural awareness 
celebration, celebration of learning, education in the new 
millennium, story telling festival, kid power, Orientation, open 
house 


34 


Events 

recognizing 

individuals 


Students (student of the week, birthdays, Honors Day, n=10) 
Grandparents (n=5) 

Special people, volunteers, guests (n=5) 


20 


School 

workers 


Volunteers, volunteer workshops 


5 



The Reading/Language Development strand (see Table 5) had 2 items on the 2000 
survey. The wording of item 1 was changed from ‘implement Kentucky Reads’ to ‘implement 
literacy program’ for this survey to reflect the change in strategy regarding reading. Five 
principals reported full implementation on their reading program. Nine schools have 
professional development in place for their reading programs (see Table 5, item 14). Eleven of 
the 12 principals made comments about what reading program they were using. Their comments 
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listed in Table 5 show that they were using a variety of reading programs. The general rating on 
Reading/Language Development would be moderate. The superintendent's rating of 
implementation of Reading/Language Development was 4 out of a possible 5. 



Table 5 

Number of schools (n = 12) at the end of year three reporting at each level of implementation for 
Reading/Language Development survey items where 0 was no implementation, 1 was very little 
implementation, and 5 was fully implemented and functioning. Principals’ comments on 
Reading/Language Development items are included. 







Level of implementation 




Strand 


Survey items 


0 1 


2 


3 


4 


5 


Read/Lang. 

Development 


1 . Implement literacy program 


0 0 


1 


2 


4 


5 




14. Professional development 


0 0 


0 


0 


3 


9 



Comments Item: 

1. - Reading and Writing component in Consolidated Plan 

- Purchase Breakthrough to literacy for K, Benchmark @1, Scholastic 
1-4, Trade books @5 

- use phonics, phonemic awareness, Benchmark, AR and STAR, 
SSRW and textbook series 

- earobics-Lab settings for 2000-01, use of speech pathologist for 
language development and phonemic awareness screening 

- Linda Mood, Benchmark, Fast Forward 

- our school is working toward the standards set forth in Reading and 
Writing Grade by Grade. Reading and Writing are components in 
Our Consolidated Plan 

- Our test scores in reading are high with no novice 4 th grade readers. 
We are waiting for district plan, but we have developed internal 
model: 1) assess, 2) small group instruction, 3) intervention for 
problems, 4) remediation strategies for kids not progressing, 5) P.D. 

- we are using many strategies that are in the new program-Fast 
Forward, Linda Mood, Testing for phonemic awareness K and PI, 
reading inventory test K-3, etc. 

- implemented breakthrough to literacy in 6 classrooms last year 

- Primary: benchmark, Open Court modeled writing, accelerated 
reader. Intermediate: Open Court, novels, Accelerated reader 

- Open Court, Reading Ren 
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Table 6 contains the reported implementation for the Foreign Language strand. 

Item 9 was reworded for this survey from ‘expose PI to second year curriculum’ to ‘implement 
second language curriculum in 1-5’ and was reported to be fully implemented by 5 of the 12 
schools. Ten schools reported full implementation on the kindergarten curriculum (item 5). 
Three of the 12 schools reported full implementation for item 4 (professional development in 
foreign language). The overall rating of implementation for Foreign Language would seem to be 
moderate. One principal commented on item 4 and that comment is included in Table 6. The 
superintendent’s rating of implementation in Foreign Language was a 3 out of a possible 5. 



Table 6 

Number of schools (n = 12) at the end of year three reporting at each level of implementation for 
Foreign Language survey items where 0 was no implementation, 1 was very little 
implementation, and 5 was fully implemented and functioning. Principals’ comments on Foreign 
Language items are included. 







Level of implementation 




Strand 


Survey items 0 


1 


2 


3 


4 


5 


Foreign 

Language: 


4. Professional development 2 


1 


4 


2 


0 


3 




5. Teach simple vocabulary in K 0 


0 


0 


2 


0 


10 




9. Second language curriculum 1-5 0 


0 


0 


5 


2 


5 


Comments 


Item: 

4. -changed to Para Ti, much more successful 













There were six items pertaining to Health/Emotional Health on the survey (see Table 7). 
Item 1 (provide school nurses as resources permit) was reported fully implemented by seven 
schools. There was some variation in amount of time different schools had nurses available. 
Principals' reports ranged from one day a week (n=3) to five days a week (n=l). Five schools 
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had a nurse two days per week. The remaining three schools reported having a nurse 1.5, 2.5, and 
3 days, respectively. Two principals reported that they had a medical technician for some part of 
the week in addition to the nurse. While all schools had nursing services at least one day per 
week, nine of the 12 schools actually have nurses more than one day per week. Nevertheless, 
only seven principals considered item 1 (provide nurses as resources permit) fully implemented. 

The remaining items were reported to be low or moderate in implementation. At least 
half the schools reported full implementation on items 10 (assess counseling staff expertise) and 
23 (provide nutrition education). Item 25 was reworded for this survey from ‘implement a 
walking program’ to ‘implement some form of fitness program’ and five schools reported full 
implementation of that item. The principals’ comments on Health/Emotional Health items 
appear in Table 7. Implementation would be considered low for this strand. The superintendent's 
rating of implementation in Health/ Emotional Health was 4 out of a possible 5. 

For the Music strand, full implementation by all schools was reported for item 3 (begin 
keyboarding program in labs). More than half the schools reported full implementation on an 
additional 14 items (see Table 8). There was much variation in reported implementation of 
Music strand items. At least one school reported relatively low levels (0, 1, or 2) of 
implementation for 9 of the Music items. Two principals changed item 1 1 (provide three 30 min. 
periods of instruction per week) to read ‘two 25 minute segments.’ Other comments by 
principals are included in Table 8. The survey items indicate that implementation of Music 
would be considered moderate. However, the superintendent's rating of implementation in 




Music was 5 out of a possible 5. 
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Table 7 

Number of schools (n = 12, unless otherwise noted) at the end of year three reporting at each 
level of implementation for Health/Emotional Health survey items where 0 was no 
implementation, 1 was very little implementation, and 5 was fully implemented and functioning. 
Principals’ comments on Health/Emotional Health items are included. 



Level of implementation 



Strand 


Survey items 


0 


1 


2 


3 


4 


5 


Health/Emot 

Health 


1 . Employ nurses as resources permit 


0 


0 


1 


3 


1 


7 




10. Assess counseling expertise 


0 


0 


0 


2 


1 


9 




23. Provide nutrition education 


0 


0 


0 


3 


3 


6 




25. Fitness program n=ll 


0 


2 


2 


0 


2 


5 




26. Professional development 


0 


2 


3 


3 


0 


4 




27. Incentives from community sponsors 


5 


2 


1 


2 


0 


2 


Comments 


Item: 

10. - Vande is a great help 
- Second Step program 

26. -curriculum alignment for vocational/practical living 
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Table 8 

Number of schools (n = 12, unless otherwise noted) at the end of year three reporting at each 
level of implementation for Music survey items where 0 was no implementation, 1 was very little 
implementation, and 5 was fully implemented and functioning. Principals’ comments on Music 
items are included 



Level of implementation 



Strand 


Survey items 


0 


1 


2 


3 


4 


5 


Music: 


1 . Allocate space 


0 


0 


0 


0 


2 


10 




2. Allocate additional space 


1 


0 


0 


1 


1 


9 




3. Begin keyboarding program in labs 


0 


0 


0 


0 


0 


12 




4. Assess necessary equipment 


0 


0 


0 


1 


1 


10 




5. Obtain Orff instruments 


1 


3 


0 


3 


0 


5 




6. Purchase glockenspiels n=l 1 


3 


1 


0 


2 


1 


4 




7. Integrate curriculum 


0 


0 


0 


1 


3 


8 




8. Begin MIE program 


0 


0 


0 


2 


0 


10 




9. Coordinate curriculum 


1 


0 


0 


2 


1 


8 




10. Integrate into regular curriculum 


0 


0 


1 


1 


3 


7 




1 1 . Three 30 min. per week 


1 


0 


0 


2 


5 


4 




12. Classify according to skills 


8 


2 


0 


0 


1 


1 




13. Establish goals 


0 


0 


1 


1 


1 


9 




14. Assure success every student 


2 


1 


2 


3 


1 


3 




15. Encourage teachers to train 


0 


0 


0 


1 


1 


10 




16. Inservice on music 


0 


0 


0 


2 


5 


5 




20. Recognize emerging talents 


0 


0 


0 


2 


3 


7 




21. After school for at-risk n=ll 


0 


0 


1 


2 


2 


6 




22. Bring musicians 


0 


0 


0 


2 


2 


8 




23. Movement in preschool n=9 


0 


0 


0 


1 


2 


6 




24. Keyboards in pre & K 


0 


2 


0 


2 


2 


6 



Comments Item: 



6. - percussion ensemble meets after school 
8. - all grades 

12. - will not work with our schedule 

16. - Socratic seminar on Brain research 

17. - district chorus 

20. - choir, talent show, percussion group 
24. - go much further 

- (no preschool) K program more academic than this (n=2) 
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There were seven Thinking Skills items on the 2000 survey (see Table 9). Eight 
principals reported that chess materials had been purchased (item 5), and half the schools have 
had training in Talents (item 1). Less than half the schools reported full implementation of the 
remaining 5 items (incorporate chess into the curriculum, hang posters of Bloom's taxonomy, 
train mentors in Talents, use technical assistance days for development of specific curriculum, 
and explore the feasibility of library cards), which would result in a low overall rating of 
implementation on Thinking Skills. The superintendent's rating of implementation in Thinking 
Skills was 3 out of a possible 5. 



Table 9 

Number of schools (n = 12, unless otherwise noted) at the end of year three reporting at each 
level of implementation for Thinking Skills survey items where 0 was no implementation, 1 was 
very little implementation, and 5 was fully implemented and functioning. Principals’ comments 
on Thinking Skills items are included 



Level of implementation 



Strand Survey items 0 1 2 3 4 5 



Thinking 

Skills 


1 . Train teachers in Talents 


0 


0 


0 


4 


2 


6 




5. Purchase chess materials 


0 


0 


0 


1 


3 


8 




6. Explore library card feasibility n=10 


5 


2 


0 


1 


0 


2 




7. Posters of Bloom's Taxonomy n=10 


1 


0 


3 


3 


0 


3 




8. Train mentor in Talents n=l 1 


8 


1 


0 


1 


0 


1 




9. Technical assistance days 


2 


1 


0 


2 


2 


5 




10. Incorporate chess 


0 


0 


0 


3 


4 


5 


Comments 


Item: 

1 . - keep getting new staff as school grows 
















7. - we use flip charts with question stems in 


every class 












9. - we are working on integrating higher level thinking across 


all subjects 






10. - most teachers use chess, but primarily as a center activity 
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F.nd of Year Three Implementation Survey: Consolidatio n into three levels: 

In order to compare implementation over time, the self reported implementation 
responses by principals were classified as 'none' (items rated 'O' by participants), 'some' (items 
rated T - '4'), or 'full' (items rated '5'). Items on the 2000 principal survey are listed and 
classified in this way in Table 10 (strands for the Arts and Family Involvement), Table 1 1 
(Reading/Language Development, Foreign Language, and Health/Emotional Health strands), and 
Table 12 (Music and Thinking Skills strands). 

Three of the five items pertaining to the Arts strand were reported as fully implemented 
(see Table 10), and items 5 (instruction in the 4 th and 5 th grades) and 6 (professional 
development) were fully implemented by 10 and 7 schools, respectively. For the actual number 
of schools reporting implementation of the Arts strand at each level, see Table 2 above. In the 
Family Involvement strand (Table 10), four items (1, 3, 19, and 22 ) out of the total of 21 items 
were reported as fully implemented by at least 10 of the 12 schools. Ten additional items were 
reported to be fully implemented by at least half the schools. Less than half the principals 
reported full implementation on the remaining seven items. The actual number of schools 
reporting implementation of the Family Involvement strand at each level is presented in Table 3 
above. 

Five schools reported full implementation of their literacy program, and the other seven 
reported some implementation. Nine schools reported full implementation of professional 
development for reading and the remainder reported some implementation (see Table 11). For 
the number of schools reporting implementation of Reading/Language Development strand items 
at each level, see Table 5 above. All schools reported some implementation of the Spanish 
program at the kindergarten level (item 5), and 10 schools reported full implementation (see 
Table 1 1). Five of the 12 schools reported second language curriculum for the upper grades to 
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be fully implemented and the remainder reported it partially implemented. Some professional 
development for Spanish was reported by 10 of the 12 schools. For the number of schools 
reporting implementation of the Foreign Language strand at each level, see Table 6 above. 



Table 10 



Number of schools (n = 12, except where noted) at the end of year three reporting no 
implementation, some implementation, or full implementation of Arts and Family Involvement 
survey items. 







Implementation Level 




Strand 




None 


Some 


Full 


Arts: 


1. Identify arts facilitator 


0 


0 


12 




2. Provide 9 arts experiences 


0 


0 


12 




3. Provide Riverpark experience for K 


0 


0 


12 




5. 20 hours instruction-grades 4 & 5 n=l 1 


0 


1 


10 




6. professional development n=ll 


0 


5 


7 


Family 










Involvement 


1 . Improve first contact, picnic 


0 


1 


11 




2. Distribute packets n=l 1 


1 


4 


7 




3. Special K registration 


0 


0 


12 




4. Personal notes to students 


0 


8 


4 




7. Homework committee/philosophy 


1 


4 


7 




8. Specific instructions re homework 


0 


3 


9 




8a. Provide syllabus if needed n=4 


0 


3 


1 




9. Reevaluate amount of homework n=l 1 


0 


6 


6 




12. Coordinator to PTO board 


2 


2 


8 




1 3 . Teacher involvement in PTO n=l 1 


0 


4 


7 




19. More opportunities for family visits 


0 


2 


10 




22. Include student planners 


0 


1 


11 




23. Improve written communication n=l 1 


0 


5 


6 




23a. Interim progress reports 


0 


4 


8 




23b. Personal notes 


0 


8 


4 




23c. Teacher articles in paper n=l 1 


2 


6 


3 




26. Schedule meetings: parents &teachers 


0 


4 


8 




27. Public relations person 


0 


5 


7 




29. Welcoming Schools criteria n=l 1 


0 


8 


3 




30. Mentoring for parent involvement 


3 


7 


2 




31. Professional development 


2 


9 


1 
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The Health/Emotional Health item (see Table 11) pertaining to providing school nurses 
(item 1), was reported as fully implemented by 7 of the 12 schools. Additional items about 
assessing counseling needs and nutrition education were reported fully implemented by 9 and 6 
schools, respectively. For the number of schools reporting implementation of the Health and 
Emotional Health strand at each level, see Table 7 above. 



Table 11 



Number of schools (n = 12, except where noted) at the end of year three reporting no 
implementation, some implementation, or full implementation of survey items for 
Reading/Language Development, Foreign Language, and Health/Emotional Health strands. 







Implementation Level 




Strand 




None 


Some 


Full 


Reading and 

Language 

Development 


1. Implement literacy program 


0 


7 


5 




14. Professional development 


0 


3 


9 


Foreign 

Language 


4. Professional development 


2 


7 


3 




5. Teach simple vocabulary in K 


0 


2 


10 




9. Second language curriculum 1-5 


0 


7 


5 


Health/ Emot. 
Health 


1 . Employ nurses as resources permit 


0 


5 


7 




10. Assess counseling expertise 


0 


3 


9 




23. Provide nutrition education 


0 


6 


6 




25. Fitness program n=l 1 


0 


6 


5 




26. Professional development 


0 


8 


4 




27. Incentives from community sponsors 


5 


5 


2 
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Table 12 

Number of schools (n = 12, except where noted) at the end of year three reporting no 
implementation, some implementation, or full implementation of Music and Thinking Skills 
survey items. 



Implementation Level 



Strand 




None 


Some 


Full 


Music 




1. Allocate space 


0 


2 


10 




2. Allocate additional space 


1 


2 


9 




3. Begin keyboarding program in labs 


0 


0 


12 




4. Assess necessary equipment 


0 


2 


10 




5. Obtain Orff instruments 


1 


6 


5 




6. Purchase glockenspiels n=l 1 


3 


4 


4 




7. Integrate curriculum 


0 


4 


8 




8. Begin MIE program 


0 


2 


10 




9. Coordinate curriculum 


1 


3 


8 




10. Integrate into regular curriculum 


0 


5 


7 




1 1 . Three 30 minutes per week 


1 


7 


4 




12. Classify according to skills 


8 


3 


1 




13. Establish goals 


0 


3 


9 




14. Assure success every student 


2 


7 


3 




15. Encourage teachers to train 


0 


2 


10 




16. Inservice on music 


0 


7 


5 




20. Recognize emerging talents 


0 


5 


7 




21. After school for at-risk n=l 1 


0 


5 


6 




22. Bring musicians 


0 


4 


8 




23. Movement in preschool n= 9 


0 


3 


6 




24. Keyboards in pre & K 


0 


6 


6 


Thinking 


Skills 


1. Train teachers in Talents 


0 


6 


t) 




5. Purchase chess materials 


0 


4 


8 




6. Explore library card feasibility n=10 


5 


3 


2 




7. Hang taxonomy posters n=10 


1 


6 


3 




8. Train mentor in Talents n=l 1 


8 


2 


1 




9. Technical assistance days 


2 


5 


5 




10. Incorporate chess 


0 


7 


5 



Responses on items for the Music strand (see Table 12) indicate that all schools have 
fully implemented the keyboarding program (items 3). Ten of the schools reported full 
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implementation on allocating space, assessing necessary equipment, beginning the MIE program, 
and encouraging teachers to take training (items, 1,4, 8, and 15). Ten additional items were 
reported as fully implemented by at least half the schools. At least one school, and as many as 
eight, reported no implementation on seven of the Music items. For the number of schools 
reporting implementation of the Music strand at each level, see Table 8 above. All Thinking 
Skills items (see Table 12) seem to have been partially implemented. More than half of the 12 
schools reported full implementation of purchasing chess equipment (item 5) and Talents 
training (item 1). For the number of schools reporting implementation of the Thinking Skills 
strand at each level, see Table 9 above. 

End of year three: comparison with original goa ls over time 

Using the original list of 1997-98 goals (see appendix for a list of these goals) as the 
standard for self reported progress, and scale values of no implementation, some implementation, 
and full implementation, third year results are presented in Table 13 for comparison with the 
three previous surveys: midway through the first year, end the first year, and end of the second 
year. The range of numbers in Table 13 (e.g., the full implementation column of year three of 
Family Involvement has a range between 1 and 11) indicate that for at least one goal, only one 
school reported full implementation and for at least one other goal, 1 1 schools reported full 
implementation. The reader is referred to previous tables to see the breakdown of 
implementation by goal. The table shows a progression in implementation by strand over the 
four measurements. 

In general, the number of schools at each successive stage of implementation increases 
from the upper left of each section of the table to the lower right, or from ’no implementation' at 
midyear the first year, to nearly 'full implementation' at the end of the third year. This is 
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especially true of the Arts strand where the number of schools reporting ‘no implementation’ of 
goals at the first midyear was 3 (upper left of Arts), and the number of schools reporting ‘full 
implementation’ at the end of three years (lower right of Arts section of the table) was 12. Also 
note that no schools reported ‘no implementation’ for the Arts strand by the end of the third year 
(lower left). With the exception of Family Involvement goals, there were no schools reporting 
'no implementation' at the end of the third year for any goal. The original 1997-98 Family 
Involvement goals were to provide a special and informative orientation or picnic at the 
beginning of school, to plan new opportunities for families to visit schools, to develop strategies 
for.improved written communication from teachers, to have teachers or counselors send home 
personal notes to all students, to add Family Involvement Coordinator and Committee to the 
PTO Board, to give specific instruction regarding homework, and to develop a homework 
philosophy through involvement of teachers, parents, administrators and students. At the end of 
the third year, two schools reported ‘no implementation’ of the goal to add a Family Involvement 
Coordinator and Committee to their PTO Boards and only one school reported ‘no 
implementation’ of the goal to develop a homework policy. 
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Table 13 

Number of schools (n = 12, except where noted) at midyear, end of the first year, end of the 



second year, and end of the third year reporting no implementation, 
implementation of the 1997-98 goals. 


some implementation, or full 


Strand 


Number of 
goals on 
1997-98 list 


Data 

collected 

at: 


Implementation Level 
None Some Full 


Arts 


3 


midyear 


3 


8 


1 






year one n = 1 1 


0 


0-1 


10-11 




(1,2,3) 


year two 


0 


0-1 


11-12 






year three 


0 


0 


12 


Family 


7 


midyear 


2 


10 


0 


Involvement 




year one n = 1 1 


0-3 


1-5 


2-10 




(1,4, 7, 8 


year two 


0 


1-4 


4-11 




12,19, 23) 


year three 


0-2 


1-8 


1-11 


Foreign 


1 


midyear 


3 


9 


0 


Language 




year one n = 1 1 


1-4 


2-3 


4-7 




(5) 


year two 


0 


0 


12 






year three 


0 


2 


10 


Health/ 


4 


midyear n = 1 1 


0 


0 


11 


Emotional 




year one n = 1 1 


0-2 


1-2 


1-10 


Health 


(1,10,23,25) 


year two 


0-9 


1-2 


1-9 






year three 


0 


3-6 


5-9 


Music 


4 


midyear 


0 


0 


12 






year one n = 1 1 


0 


0 


11 




(1,3, 10, 15) 


year two 


0 


0 


12 






year three 


0 


0-5 


7-12 


Reading and 


1 


midyear 


1 


4 


7 


Language 




year one n = 1 1 


0-2 


1-5 


3-10 


Development 


(1) 


year two 


0 


1-2 


9-11 






year three 


0 


7 


5 


Thinking 


2 


midyear 


2 


8 


1 


Skills 




year one n = 1 1 


3-4 


2-5 


3-5 




(1,5) 


year two 


0 


0-5 


5-10 






year three 


0 


4-6 


6-8 
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On the other end of the implementation spectrum, however. Table 13 seems to indicate 
that even at the end of the third year, most of the strands are not yet fully implemented in terms 
of the 1997-98 goals for implementation (see lower right of each section of Table 13). Family 
Involvement had between one and four schools reporting ‘some implementation’ at the end of 
year two, and between one and eight at the end of year three. At the end of year two Family 
Involvement had between four and 1 1 schools reporting ‘full implementation’ on some goals and 
at the end of year three, between one and 1 1. Foreign Language had 12 schools reporting full 
implementation at the end of year two, but only 10 at the end of year three. Health/Emotional 
Health had between one and nine at the end of year two and between five and nine at the end of 
year three. Music had 12 schools reporting ‘full implementation’ at the end of year two, but 
between seven and 12 at the end of year three. Reading and Language Development had 
between nine and 1 1 at the end of year two, and only five at the end of year three. Finally, 
Thinking Skills had between five and ten at the end of year two and between six and eight at the 
end of year three. With respect to the original list of 1997-98 goals, in some cases at least, there 
seems to be a shift at the end of the third year away from ‘full implementation’ in spite of the 
fact that the general tendency is for gradual increments of progress over the three year time 
frame. Possible explanations for these results will be reviewed in the discussion that follows the 
results. 

When percentage of full implementation of the 1997-98 goals is used as the standard, 
implementation at the end of year three in the Arts strand would be considered complete (100%); 
implementation in Music and Foreign Language would be considered high (81% and 83%, 
respectively); implementation in Family Involvement (65%), Thinking Skills (58%), and 
Health/Emotional Health (58%) would be considered moderate, and implementation in Reading 
and Language Development (42%) would be considered low. The reader is cautioned to take 
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into consideration that percentages are based on few goals (and in some cases only one goal) and 
do not represent accurately what has been accomplished in any of the strands overall. See the 
discussion on the 1997-98 goals in the next section. 

Summary 

In summary, when considering all items on the 2000 survey and looking at the overall 
self report data for the 12 schools reporting, two strands (Thinking Skills and Health/Emotional 
Health) appear to have been relatively low in implementation. For these two strands, there were 
no items for which at least 10 of the 12 schools reported full implementation and several items 
(three out of six for Health/Emotional Health and five of the seven for Thinking Skills) had less 
than half the schools reporting full implementation. The superintendent's ratings of these two 
strands were moderately high (Health/Emotional Health) and moderate (Thinking Skills). Using 
a similar standard. Music would be considered moderate in implementation since five of the 21 
items had 10 to 12 schools reporting full implementation. Fifteen of the 21 items had at least 
half the schools in full implementation, and several items had implementation ratings at the 
lower end of the scale. The superintendent rated Music as high in implementation. Family 
Involvement could be said to have moderate implementation overall since four of the 21 items 
were reported to be in full implementation by at least 10 schools, and 14 items were rated as 
fully implemented by at least half the schools. The superintendent's rating of Family 
Involvement was moderately high. 

The Foreign Language strand appears to have been moderate in implementation. Ten 
principals reported full implementation for one of the three Foreign Language goals, and less 
than half the principals reported full implementation of the other two goals. The superintendent's 
rating was also moderate for Foreign Language. Reading and Language Development would be 




26 



Graduation 2010-26 

rated as moderate for the third year since only nine schools reported full implementation of one 
goal and less than half reported full implementation of the other goal. The superintendent’s 
rating was moderately high for Reading and Language Development. Finally, the strand for the 
Arts appears to have been high in implementation according to the survey of principals. 

Between ten and 12 schools reported full implementation on four of the five goals in the Arts, 
and seven of the 12 schools reported full implementation on the fifth goal. The superintendent 
rated the Arts moderately high in implementation across the district. 

If the measurement of implementation is limited to the original 1997-98 goals, strands 
high in implementation would be the Arts, Foreign Language and Music. Family Involvement, 
Thinking Skills, and Health/Emotional Health would be classified as moderate, and 
Reading/Language Development would be low. 

Discussion 

The report on implementation at the end of year two (O’Phelan et al., 1999b) discussed 
several factors that complicate the interpretation of data from implementation surveys like the 
ones used here. Since these factors have similar effects on the third year data presented in this 
report, they will be briefly discussed again. The classification of the implementation of each 
strand as low, moderate or high is arbitrary because strands had unequal number of items on the 
surveys. Original plans for implementation were much more detailed for some strands than for 
others, such that the judgment of degree of implementation might be based on one item in one 
strand but 21 items in another. Such classifications tended to handicap strands with many items 
even though much had been accomplished in those strands. This problem was further 
complicated by the fact that some items on the previous survey have been eliminated, and some 
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new items added to the 2000 survey. In any case the classification of level of implementation is 
necessarily subjective. 

A related problem is that some implementation steps are much more complicated and 
involved than others. Consider the comparison of two Music strand steps (allocate space for 
keyboard labs, and integrate music into the curriculum). Full implementation of the first of these 
merely indicates that someone decided where to put the keyboard lab, whereas full 
implementation of the latter could come only after cooperative efforts of many people, and may, 
in fact, never be achieved because it will be affected by changes in the times, in the children, and 
in the curriculum itself. Assigning these two implementation steps equal weight in the 
evaluation process is specious. Evaluation should be more than tallying frequencies. 

The scope of the program requires much cooperation and planning, and it was never 
intended to be implemented all at once. Implementation has not been required of any school, and 
decisions about what and when to implement were left up to the individual school. Likewise, 
principals were not given definitions of different levels of implementation, so what one principal 
considered 'full implementation' might have been considered only partial implementation by 
another principal. This may explain the finding that ten schools reported a nurse in place at the 
end of the first year, nine reported full implementation at the end of the second year, and only 
seven reported full implementation of that goal at the end of year three (see Table 7). When 
principals were asked how may days a week they had a nurse, their responses ranged from one 
day to five days, and there were no responses less than one day. Nevertheless, seven of the 12 
did not report full implementation. One might speculate that some principals reported less than 
full implementation because they regard one day a week as insufficient and would like to 
increase the number of days they have a nurse. If ratings were based on what they would like to 
have, rather than what they actually have in place already, then the area of Health/Emotional 
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Health is really better implemented than it appears in this report. The reader is reminded that 
ratings were based on what principals reported, not on outside criterion measures of the survey 
items. Respondents’ subjective interpretation of survey items might also explain the fact that the 
superintendent’s general rating of implementation for each strand did not exactly agree with 
ratings derived from the responses of principals. A comparison of superintendent’s and 
principals’ ratings is presented Table 14 in the appendix. 

Another problem associated with the surveys was the imperfect alignment between the 
survey questions and the list of goals to be evaluated. This was especially true for surveys after 
the. first year (1997-98) because there has been no revision of the list of goals to be accomplished 
since the 1997-1998 list of goals. Administrative personnel were asked to screen the items for 
the 1999 and 2000 surveys to make them more appropriate measures of progress made during the 
those years, but no new list of objectives has been agreed upon by the steering committee. For 
this reason, two different standards of comparison were used in this report, the original 1997-98 
goals, and all 2000 survey items. 

Each of these methods has its problems. The entire list of items has changed somewhat 
over the three years of the project, so the measuring instrument is not the same one used three 
years ago. Some new goals have been added along the way and several items have been 
removed or modified because they no longer matched the program in practice. On the other 
hand, the list of 1997-98 goals has not changed, so the items for those goals have been constant 
over the three years. Those goals, however, may no longer be relevant for measurement of 
implementation since the emphasis of the project may have changed over the three years. 

Items intended to measure the implementation of the 1997-98 goals have been numbered 
the same and worded the same on all surveys with the exception of the first goal for Reading and 
Language Development which previously said “implement Kentucky Reads Grant” and now 
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says “implement literacy program.’ These items, although constant on all surveys, have not been 
marked in any way or listed together so that respondents would be able to recognize them as 
different from other survey items. In spite of this, responses seem to indicate less, or at least 
more variable, implementation in year three than in year two (see Table 13, p. 23). This 
phenomenon could be explained in the case of Reading and Language Development by the fact 
that the Kentucky Reads has been replaced by other reading programs and it may be that these 
have not been fully installed or implemented thus far. 

It is not as easy to explain the fact that at the end of year two all 12 schools reported full 
implementation of the Music and Foreign Language strands, but less than 12 reported full 
implementation at the end of year three (see Table 13). One explanation is that the same survey 
items were interpreted differently on the two occasions (end of year two and year three). 

Another possibility is that reliability factors (e.g., whether or not the principals were in a hurry, 
whether or not they were able to give the survey adequate attention, or even whether the same 
person at the school responded for the two years) caused the pattern of results. Still another 
possibility is that some activities carried out the second year of the project have been 
discontinued (e.g., simple vocabulary was taught to kindergartners last year, but was not this year 
(see Table 1 1) or that space was allocated for the music lab last year, but not this year (see Table 
12), although this explanation seems less plausible. 

Perhaps program goals have changed significantly over the three years, and the list of 
goals on the survey no longer reflects what is really being done in the schools on each strand, in 
spite of the fact that school personnel screened survey items. If the goals set for the first year of 
the project (1997-98 goals) are still relevant, one would expect that progress in implementation 
of those goals would be faster and more complete than for the entire set of original goals formed 
by the steering committee for the project. The comparison of ratings using the 1997-98 goals 
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and all items on the 2000 survey would seem to confirm that prediction. Ratings using the 1997- 
98 goals are the same as ratings on all 2000 items for the Arts (high and high), and for Family 
Involvement (moderate and moderate). Ratings using the 1997-98 goals are higher than ratings 
on all goals for Foreign Language (high and moderate), Music (high and moderate), Thinking 
Skill (moderate and low), and Health/Emotional Health (moderate and low). Comparable ratings 
on Reading and Language Development (low and moderate) did not follow this pattern, but the 
overall rating for Reading was based on only two items, and the 1997-98 goals rating was based 
on a single item. 

Consideration of factors mentioned here would make ratings of implementation seem 
tenuous, at best, especially for those strands where the number of items on the survey has 
dwindled to one or two. The original plans submitted by subcommittees for each strand included 
successive steps to be implemented over time. In some cases these plans provided for self study 
and setting new goals for the future. It is unlikely that the implementation of any of these . 
original plans could be measured using so few items. The programs implemented have 
continued to grow and change, and perhaps the survey for implementation no longer accurately 
reflects what is happening. Survey items tied to some of these original steps have been removed 
from the survey if those steps are not being implemented. On the other hand, there is no new list 
of goals from which to make new items to more accurately reflect current practices. The 
program could be much better served if the data collection instrument accurately reflected 
current goals and current practices in the district. This could be accomplished by updating goals 
and by modifying the survey accordingly. The evaluation team hopes to be allowed to study the 
program and describe it in more depth than the implementation survey data permit. 
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Table 15 (continued) Survey items included on the 2000 survey of principals, classified by strand: (*) denotes a new item 
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Table 15 (continued) Survey items included on the 2000 survey of principals, classified by strand: (*) denotes a new item 
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